Methods and systems for identifying user action

ABSTRACT

The embodiment of the present disclosure provides a method and a system for identifying a user action. The method and system may obtain user action data collected from a plurality of measurement positions on a user, the user action data corresponding to an unknown user action, identify that the user action includes a target action when obtaining the user action data based on at least one set of target reference action data, the at least one set of target reference action data corresponding to the target action, and send information related to the target action to the user.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of International Patent ApplicationNo. PCT/CN2022/074379, filed on Jan. 27, 2022, which claims priority ofInternational Patent Application No. PCT/CN2021/081931, filed on Mar.19, 2021, the contents of each of which are entirely incorporated hereinby reference.

TECHNICAL FIELD

The present disclosure relates to the technical field of wearableapparatus, and in particular, to methods and systems for identifying auser action.

BACKGROUND

With people's attention to scientific sports and physical health,fitness motion monitoring device is developing greatly. At present, themain way for the motion monitoring device to monitor the user's actionis to analyze the user's action data based on the reference action datawhen the type of the user's action is known, so as to monitor whetherthe user's action is standardized. Therefore, in a practical applicationscenario, the user needs to inform the motion monitoring device of thefitness action type in advance before exercising, so that the motionmonitoring device can select the reference action data of the actiontype to make accurate monitoring of the user action. For the user,before each fitness action, the action type must be informed to themotion monitoring device, which leads to a poor user experience.Moreover, the existing motion monitoring devices monitor the user'saction in a non-real time manner, which results in that the user canonly receive information related to the fitness action after completingthe fitness action, which also leads to a poor user experience.

Therefore, it is necessary to provide a method and a system foridentifying a fitness action of a user in real-time without the need forthe user to input the action type in advance.

SUMMARY

The present disclosure discloses a method of identifying a user action.According to one aspect of the present disclosure, the method mayinclude the following operation. User action data collected from aplurality of measurement positions on a user may be obtained, the useraction data may correspond to an unknown user action. The user actionincludes a target action when obtaining the user action data may beidentified based on at least one set of target reference action data,the at least one set of target reference action data may correspond tothe target action. Information related to the target action may be sentto the user.

In some embodiments, the identifying that the user action includes atarget action may include the following operations. A plurality of setsof candidate reference action data may be obtained, wherein each set ofcandidate reference action data corresponds to at least one referenceaction. A two-level screening operation on the plurality of sets ofcandidate reference action data may be performed based on the useraction data, the two-level screening operation may include a combinationof a difference degree-based screening operation and a probability-basedscreening operation. The user action includes the target action based ona result of the two-level screening operation may be determined.

In some embodiments, the identifying that the user action includes atarget action may include the following operations. A plurality of setsof reference action data may be obtained, wherein each set of referenceaction data corresponds to at least one reference action. Each set ofreference action data in turn from the plurality of sets of referenceaction data may be selected as candidate reference action data. At leastone difference degree may be determined by comparing at least onesegment of action identification sub-data of the candidate referenceaction data with the corresponding user action sub-data segment bysegment. A comprehensive difference degree may be determined byweighting and summing the at least one difference degree.

In some embodiments, each set of reference action data may include Mpieces of reference action sub-data, each piece of the reference actionsub-data may include at least one segment of action identificationsub-data, and M may be an integer greater than 1. Action identificationsub-data of the M pieces of reference action sub-data may form integralaction identification data, and each segment of action identificationsub-data may correspond to at least a portion of the reference action onat least one measurement position of the plurality of measurementpositions.

In some embodiments, the determining at least one difference degree bycomparing at least one segment of action identification sub-data of thecandidate reference action data with the corresponding user actionsub-data segment by segment may include the following operations. Asliding window with a preset length on each piece of the actionidentification sub-data may be selected, the sliding window may includea data segment of the user action data collected in a preset timeinterval. For the sliding window at a current moment, the differencedegree between the data segment and the corresponding actionidentification sub-data may be determined.

In some embodiments, the identifying that the user action includes thetarget action further may include the following operations. A value ofthe comprehensive difference degree is greater than a first preset valuemay be determined. The sliding window may slide to a next data segmentwith a preset step size, and the comparison may be repeated.

In some embodiments, a data collection time length corresponding to thedata segment in the sliding window may be negatively correlated with auser action speed.

In some embodiments, the preset step size may satisfy one or morefollowing conditions. The preset step size may be positively correlatedwith a magnitude of a value of the comprehensive difference degree at aprevious moment. The preset step size may be positively correlated witha variation trend of the value of the comprehensive difference degree.

In some embodiments, the data segment may include a plurality of useraction data points. The determining at least one difference degree bycomparing at least one segment of action identification sub-data of thecandidate reference action data with the corresponding user actionsub-data segment by segment may include the following operations. Atarget comparison data interval may be selected from the actionidentification sub-data, wherein the target comparison data intervalincludes a plurality of identification data points. The data segmentaccording to a plurality of scales may be adjusted to obtain a pluralityof adjusted data segments. A difference degree between the actionidentification sub-data and each adjusted data segment of the pluralityof adjusted data segments may be determined respectively. A minimumdifference degree between the action identification sub-data and thedata segment may be determined.

In some embodiments, the determining at least one difference degree bycomparing at least one segment of action identification sub-data of thecandidate reference action data with the corresponding user actionsub-data segment by segment may include the following operations. Adistance matrix [D_(ij)] may be determined, wherein D_(ij) denotes adistance between an i-th data point of a target comparison data intervaland a j-th data point of the data segment. A shortest distance path ofthe distance matrix may be determined, wherein the shortest distancepath may satisfy the following operations. A start point of the shortestdistance path may be in the first line of the [D_(ij)], two adjacentpoints on the shortest distance path may be adjacent in the distancematrix, a next point on the shortest distance path may be to the right,below or right below a previous point, an end point of the shortestdistance path may be in a last line of the [D_(ij)], the shortestdistance path may have a smallest regularization cost, wherein theregularization cost is determined by distances of points on thecorresponding shortest distance path of the distance matrix, and hedifference degree may be related to the regularization cost.

In some embodiments, if the first data point of the data segment may bedetermined to be a data point where the user action starts, the startpoint of the shortest distance path may be a distance D₁₁ between thefirst point of the data segment and the first point of the targetcomparison data interval.

In some embodiments, if the last data point of the data segment may bedetermined to be the data point where the user action ends, the endpoint of the shortest distance path may be a distance D_(mn) between thelast point of the data segment and the last point of the targetcomparison data interval.

In some embodiments, the identifying that the user action includes thetarget action further may include the following operations. N pieces ofsecond-level candidate reference action data may be selected from theplurality of sets of reference action data. A value of the comprehensivedifference degree of the second-level candidate reference action datamay be less than a first preset value, and N may be an integer greaterthan 1. N distances between the user action data and the N pieces ofsecond-level candidate reference action data may be calculatedrespectively. N probability values may be calculated based on the Ndistances respectively. The second-level candidate reference action datawhose probability value is greater than a second preset value may beselected as the target reference action data. A reference actioncorresponding to the target reference action data may be determined asthe target action.

Another aspect of the present disclosure discloses a system foridentifying a user action. The system for identifying the user actionmay include the following operations. At least one storage medium, theat least one storage medium may store at least one instruction set forobtaining user action data during the user's motion. At least oneprocessor, in communication with the at least one storage medium,wherein when the system is running, the at least one processor may readthe at least one instruction set and execute the above method and themethod for identifying a target action disclosed in the embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure may be further described by way of exemplaryembodiments, which may be described in detail by means of theaccompanying drawings. These embodiments are not limiting, and in theseembodiments, the same numbers refer to the same structures, wherein:

FIG. 1 illustrates a schematic diagram of an application scenario of amotion monitoring system according to some embodiments of the presentdisclosure;

FIG. 2 illustrates a schematic diagram of exemplary hardware and/orsoftware components of a wearable apparatus according to someembodiments of the present disclosure;

FIG. 3 illustrates a schematic diagram of exemplary hardware and/orsoftware components of a computing device according to some embodimentsof the present disclosure;

FIG. 4 illustrates an exemplary structural diagram of a wearableapparatus according to some embodiments of the present disclosure;

FIG. 5 is a flowchart illustrating an exemplary process for determininga target action according to some embodiments of the present disclosure;

FIG. 6 illustrates an exemplary coordinate system diagram during auser's motion according to some embodiments of the present application;

FIG. 7A shows an exemplary segment of action identification data inreference action data and a curve of a segment of the user actionsub-data collected by the sliding window in the user action data on thetime axis according to some embodiments of the present disclosure;

FIG. 7B illustrates a distance matrix and a shortest distance path fromthe upper left corner to the lower right corner of the distance matrixaccording to some embodiments of the present disclosure;

FIG. 7C illustrates a schematic diagram of determining the comprehensivedifference degree through a sliding window when the user action dataincludes a plurality of user action sub-data according to someembodiments of the present disclosure.

DETAILED DESCRIPTION

In order to illustrate technical solutions of the embodiments of thepresent disclosure more clearly, the following briefly illustratesdrawings in the illustration of the embodiments. Drawings in thefollowing illustration are merely some examples or embodiments of thepresent disclosure. For those skilled in the art, the present disclosuremay be applied to other similar scenarios in accordance with thedrawings without creative works. Unless obviously obtained from thecontext or the context illustrates otherwise, the same number in thedrawings refers to the same structure or operation.

It should be understood that “system,” “device,” “unit,” and/or “module”used herein are a method for distinguishing different components,elements, members, portions, or assemblies of different levels. However,if other words may achieve the same purpose, the words may be replacedby other expressions.

As used in the disclosure and the appended claims, the singular forms“a,” “an,” and “the” include plural referents unless the content clearlydictates otherwise. In general, the terms “comprising” and “including”only prompt steps and elements that are explicitly identified, and thesesteps and elements do not constitute an exclusive list. Methods ordevice may also include other steps or elements.

Flowcharts are used in the present disclosure to illustrate theoperations performed by the system according to some embodiments of thepresent disclosure. It should be understood that the front or rearoperations may not be necessarily performed exactly in order. On thecontrary, each step may be performed in reverse or simultaneously. Atthe same time, other operations may also be added to the procedures, ora certain step or several steps may be removed from the procedures.

The present disclosure provides a target action determination system. Aset of instructions stored in a storage medium in the target actiondetermination system may be executed to obtain user action data duringthe user's motion. The target action determination system may be appliedto a wearable apparatus (e.g., a clothing, a wristband, and a helmet), amedical testing device (e.g., an electromyographic (EMG) tester), afitness device, etc. After the user wears the device, sensors on thedevice may be attached to a plurality of measurement positions on theuser's body, so the user's action data may be collected by the sensorson the device. After the sensors collect the user action data, theprocessor in the target action determination system may communicate withthe storage medium to access or read the instructions stored in thestorage medium, etc. When the target action determination system isrunning, the processor may access reference action data stored in thestorage medium with known action content. Based on the reference actiondata of these known action contents, the system may perform a targetaction identification on the user action data whose action contents areunknown. After determining the target action, the system may send acontent related to the target action to the user.

In some embodiments of the present disclosure, the system may performthe target action identification on the user action data immediately orwithin a predetermined time, the predetermined time may be a short time,such as 0.1 seconds or 0.5 seconds. In this way, the system may realizereal-time identification of the user action data, and the user mayimmediately receive the related content about the action afterperforming the action.

In some embodiments of the present disclosure, the user action data mayalso be obtained in other ways without being collected by sensors ondevices such as a wearable apparatus (e.g., a clothing, a wristband, ahelmet), a medical detection device (e.g., an EMG tester), a fitnessdevice, etc. For example, user images in a video may be analyzed by anartificial intelligence algorithm to obtain action data of severalmeasurement positions on the user's body. In a word, as long as the useraction data may be obtained in real time, the method and system of thepresent disclosure may be configured to determine the target action.

The storage medium may include a propagated data signal having acomputer program code embodied therein, for example, at baseband or aspart of a carrier wave. The propagated signal may take variousmanifestations, including electromagnetic, optical, etc., or a suitablecombination. The computer storage medium may be any computer-readablemedium other than computer-readable storage medium that may communicate,propagate, or transmit a program for use by coupling to an instructionexecution system, an apparatus, or a device. The program code on thecomputer storage medium may be transmitted over any suitable medium,including radio, a cable, a fiber optic cable, RF, or the like, or acombination thereof. Specifically, the storage medium may be arandom-access memory (RAM), a read only memory (ROM), etc. Exemplary ROMmay include a mask ROM (MROM), a programmable ROM (PROM), an erasableprogrammable ROM (PEROM), an electrically erasable programmable ROM(EEPROM), a compact disc ROM (CD-ROM), a digital universal disc ROM,etc. Exemplary RAM may include a dynamic RAM (DRAM), a double ratesynchronous dynamic RAM (DDR SDRAM), a static RAM (SRAM), a thyristorRAM (T-RAM), a zero capacitance (Z-RAM), or the like.

As an example, the processor may be a multi-core processor, asingle-core processor, a distributed processor, a central processingunit (CPU), an application-specific integrated circuit (ASIC), anapplication-specific instruction processor (ASIP), a graphics processor(GPU), a physical processor (PPU), a digital signal processor (DSP), afield programmable gate array (FPGA), a programmable logic circuit(PLD), a controller, a microcontroller unit, a reduced instruction setcomputer (RISC), a microprocessor device, or the like, or anycombination thereof.

FIG. 1 illustrates a schematic diagram of an application scenario of amotion monitoring system according to some embodiments of the presentdisclosure. As shown in FIG. 1 , the system 100 for determining a targetaction (or system 100) may include a processing device 110, a network120, a wearable apparatus 130, and a mobile terminal device 140. Thesystem 100 may obtain user action data (e.g., an EMG signal, a posturesignal, stress data, and physiological monitoring data such as an ECGsignal, a respiratory rate signal, etc.) configured to characterize theuser's action, and identify which target action the action of the userbelongs to when exercising according to the user action data.

For example, the system 100 may identify an action of a user performedby the user while exercising. When the user wears the wearable apparatus130 for fitness exercise, the wearable apparatus 130 may obtain theuser's action data. The processing device 110 or the mobile terminaldevice 140 may receive and analyze the user's action data to identifythe user's fitness action, such as whether the user's fitness action isa bench press, a bicep curl, or a squat, etc., so as to send a contentrelated to the target action to the user, wherein the user action of theidentified action (e.g., a bench press, a bicep curl, a squat, etc.) isthe target action. Specifically, the content related to the targetaction may include the name of the target action prompted by voice orvideo, the action type of the action, the action quantity, the actiontime, the user's physiological parameter information when the userperforms the action, etc. Further, the system 100 may generate feedbackon the user's fitness action based on the analysis result of the user'sfitness action data, such as whether the user's fitness action isstandard, etc., so as to guide the user's fitness.

As another example, the system 100 may identify the user's actionperformed by the user while running. For example, when the user wearsthe wearable apparatus 130 to perform a running motion, the system 100may obtain the user's running action data and identify that the user'scurrent motion is running based on reference action data. When the userruns for too long or the running action is incorrect, the fitness devicemay feed back his/her motion status to the user to prompt the user toadjust the running action or running time.

In some embodiments, the processing device 110 may be configured toprocess information and/or data related to user motion. For example, theprocessing device 110 may receive the user's action signal (e.g., an EMGsignal, a posture signal, an ECG signal, a respiratory rate signal,etc.), and further extract feature information (e.g., the featureinformation corresponding to the EMG signal or the feature informationcorresponding to the posture signal in the action signal) correspondingto the action signal. In some embodiments, the processing device 110 mayperform specific signal processing on the EMG signal or gesture signalcollected by the wearable apparatus 130, such as signal segmentation,signal preprocessing (e.g., signal correction processing, filteringprocessing, etc.), or the like. In some embodiments, the processingdevice 110 may also determine whether the user's action is correct basedon the user's action signal. For example, the processing device 110 maydetermine whether the user's action is correct based on the featureinformation (e.g., amplitude information, frequency information, etc.)corresponding to the EMG signal. As another example, the processingdevice 110 may determine whether the user's action is correct based onfeature information (e.g., angular velocity, angular velocity direction,angular velocity acceleration, angle, displacement information, stress,etc.) corresponding to the gesture signal. As further another example,the processing device 110 may determine whether the user's action iscorrect based on the feature information corresponding to the EMG signaland the feature information corresponding to the gesture signal. In someembodiments, the processing device 110 may also determine whether theuser's physiological parameter information during the user's motionmeets the health standard. In some embodiments, the processing device110 may also issue corresponding instructions to feed back the user'smotion situation. For example, when the user is running, the system 100monitors that the user's running time is too long. At this time, theprocessing device 110 may issue an instruction to the mobile terminaldevice 140 to prompt the user to adjust the running time. It should benoted that the feature information corresponding to the gesture signalis not limited to the above-mentioned angular velocity, angular velocitydirection, angular velocity acceleration, angle, displacementinformation, stress, etc., but may also be other feature information.Any parameter information that can reflect the relative motion of theuser's body may be the feature information corresponding to the gesturesignal. For example, when the posture sensor is a strain gauge sensor, abending angle and a bending direction of the user's joint may beobtained by measuring the magnitude of the resistance in the straingauge sensor that changes with the stretched length.

In some embodiments, the processing device 110 may be local or remote.For example, the processing device 110 may access information and/ordata stored in the wearable apparatus 130 and/or the mobile terminaldevice 140 through the network 120. In some embodiments, the processingdevice 110 may connect directly with the wearable apparatus 130 and/orthe mobile terminal device 140 to access information and/or data storedtherein. For example, the processing device 110 may be located in thewearable apparatus 130 and realize information interaction with themobile terminal device 140 through the network 120. As another example,the processing device 110 may be located in the mobile terminal device140 and realize information interaction with the wearable apparatus 130through the network. In some embodiments, the processing device 110 maybe implemented on a cloud platform.

In some embodiments, the processing device 110 may process data and/orinformation related to action monitoring to perform one or morefunctions described herein. In some embodiments, the processing device110 may obtain an action signal during the user's motion collected bythe wearable apparatus 130. In some embodiments, the processing device110 may send a control instruction to the wearable apparatus 130 or themobile terminal device 140. The control instruction may control thestates of switches of the wearable apparatus 130 and sensors of thewearable apparatus 130, and may also control the mobile terminal device140 to send out prompt information. In some embodiments, the processingdevice 110 may include one or more sub-processing devices (e.g., asingle-core processing device or a multi-core processing device).

The network 120 may facilitate the exchange of data and/or informationof the motion monitoring system 100. In some embodiments, one or morecomponents of the motion monitoring system 100 may send data and/orinformation to other components of the motion monitoring system 100through network 120. For example, an action signal collected by thewearable apparatus 130 may be transmitted to the processing device 110through the network 120. As another example, a confirmation result forthe action signal determined by the processing device 110 may betransmitted to the mobile terminal device 140 through the network 120.In some embodiments, the network 120 may be any type of wired orwireless network.

The wearable apparatus 130 may refer to a garment or apparatus with awearable function. In some embodiments, the wearable apparatus 130 mayinclude, but is not limited to, an upper garment device 130-1, a pantsdevice 130-2, a wristband device 130-3, a shoe device 130-4, or thelike. In some embodiments, the wearable apparatus 130 may include Msensors, M is an integer greater than one. The sensors may obtainvarious action signals (e.g., an EMG signal, a posture signal,temperature information, a heartbeat frequency, an electrocardiogramsignal, etc.) generated during the user's motion. In some embodiments,the sensors may include, but are not limited to, one or more of an EMGsensor, a posture sensor, a temperature sensor, a humidity sensor, anelectrocardiogram sensor, a blood oxygen saturation sensor, a Hallsensor, an electrodermal sensor, a rotation sensor, or the like. Forexample, the upper garment device 130-1 may include an EMG sensorpositioned at a muscle (e.g., biceps brachii, triceps brachii,latissimus dorsi, trapezius, etc.) position of the human body, and theEMG sensor may fit the user's skin and collect an EMG signal during theuser's motion. As another example, the upper garment device 130-1 mayinclude an electrocardiogram sensor positioned near the left pectoralmuscle of the human body, and the electrocardiogram sensor may collectan electrocardiographic signal of the user. As further another example,the pants device 130-2 may include a posture sensor positioned at amuscle (e.g., gluteus maxim us, vastus lateralis, vastus medialis,gastrocnemius, etc.) position of the human body, and the posture sensormay collect a posture signal of the user. In some embodiments, thewearable apparatus 130 may also provide feedback on the user's action.For example, when an action of a certain portion of the body during theuser's motion does not meet the standard, the EMG sensor correspondingto this portion may generate a stimulation signal (e.g., currentstimulation or hitting signal) to remind the user.

It should be noted that the wearable apparatus 130 is not limited to theupper garment device 130-1, the pants device 130-2, the wristband device130-3, or the shoe device 130-4 shown in FIG. 1 . The wearable apparatus130 may also include other apparatuses used for motion monitoring, suchas a helmet device, a kneepad device, etc., which is not limited here.Any apparatus that can use the motion monitoring method disclosed in thepresent disclosure is within the scope of protection of the presentdisclosure.

In some embodiments, the mobile terminal device 140 may obtaininformation or data in the system 100. In some embodiments, the mobileterminal device 140 may receive action data processed by the processingdevice 110, and feed back an action record based on the processed actiondata. Exemplary feedback modes may include, but are not limited to, avoice prompt, an image prompt, a video presentation, a text prompt, orthe like. In some embodiments, the user may obtain the action recordduring his/her own motion through the mobile terminal device 140. Forexample, the mobile terminal device 140 may be connected with thewearable apparatus 130 through the network 120 (e.g., wired connection,wireless connection). The user may obtain the action record during theuser's motion through the mobile terminal device 140, and the actionrecord may be transmitted to the processing device 110 through themobile terminal device 140. In some embodiments, the mobile terminaldevice 140 may include a mobile device 140-1, a tablet computer 140-2, anotebook computer 140-3, or the like, or any combination thereof. Insome embodiments, the mobile device 140-1 may include a cell phone, asmart home device, a smart mobile device, a virtual reality device, anaugmented reality device, etc., or any combination thereof. In someembodiments, the smart home device may include a control device for asmart appliance, a smart monitoring device, a smart TV, a smart camera,etc., or any combination thereof. In some embodiments, the smart mobiledevice may include a smart phone, a personal digital assistant (PDA), agaming device, a navigation device, a POS device, etc., or anycombination thereof. In some embodiments, the virtual reality deviceand/or the augmented reality device may include a virtual realityheadset, virtual reality glasses, a virtual reality eyewear, anaugmented reality helmet, augmented reality glasses, an augmentedreality eyewear, etc., or any combination thereof.

In some embodiments, the motion monitoring system 100 may also includean action data presentation system 160. The action data presentationsystem 160 may be configured to process and display information and/ordata related to the user's action. For example, what kind of motion theuser is doing may be displayed, or the information and/or data may becombined with a virtual character and intuitively displayed on a userinterface of the mobile terminal device 140 to facilitate the user toview. For example, the action data presentation system 160 may receivethe user's action data. For instance, the user's action data may includean action signal such as an EMG signal, a posture signal, anelectrocardiogram signal, a respiratory rate signal, etc. As anotherexample, the user's action data may include feature information (e.g.,feature information corresponding to the EMG signal, feature informationcorresponding to the gesture signal in the action signal) obtained bythe processing device 110 performing feature processing on the actionsignal. As further another example, the user's action data may include asingle obtained after the processing device 110 performs specific signalprocessing, such as signal segmentation, signal preprocessing (e.g.,signal correction processing, filtering processing, etc.), etc. Theaction data presentation system 160 may compare the action data with thereference action data, combine the comparison result with a virtualcharacter to generate an animation of the virtual character, and sendthe generated animation to the mobile terminal device 140 for display.The reference action data may be described in detail in the followingdescriptions. For example, when the user is doing biceps curling, theaction data presentation system 160 may receive the action data of theuser when performing biceps curling, such as an EMG signal of the bicepsbrachii, an EMG signal of the trapezius, a movement posture of theforearm, a movement posture of the forearm, etc. The action datapresentation system 160 may compare the user's action data with aplurality of sets of reference action data stored in the motionmonitoring system 100 to determine that the user is performing theaction of biceps curling. Further, the action data presentation system160 may display a virtual character that is doing biceps curling, andthe user may clearly and intuitively view the user's action data or adifference between the action data and the reference action data (e.g.,a difference in the position and size of muscle forces, a difference inthe action posture, etc.) through the animation of the virtual characterto adjust the action during the motion.

In some embodiments, the action data presentation system 160 may beintegrated in the processing device 110. In some embodiments, the actiondata presentation system 160 may also be integrated in the mobileterminal device 140. In some embodiments, the action data presentationsystem 160 may also exist independently of the processing device 110 andthe mobile terminal device 140. The action data presentation system 160may be connected in communication with the processing device 110, thewearable apparatus 130, and the mobile terminal device 140 to transmitand exchange information and/or data. In some embodiments, the actiondata presentation system 160 may access information and/or data storedin the processing device 110, the wearable apparatus 130, and/or themobile terminal device 140 via the network 120. In some embodiments, thewearable apparatus 130 may connect directly with the processing device110 and/or mobile terminal device 140 to access information and/or datastored therein. For example, the action data presentation system 160 maybe located in the processing device 110 and realize informationinteraction with the wearable apparatus 130 and the mobile terminaldevice 140 through the network 120. As another example, the action datapresentation system 160 may be located in the mobile terminal device 140and realize information interaction with the processing device 110 andthe wearable apparatus 130 through the network. In some embodiments, theaction data presentation system 160 may be executed on a cloud platform,and realize information interaction with the processing device 110, thewearable apparatus 130, and the mobile terminal device 140 through thenetwork.

For the convenience of presentation, in the following descriptions, theaction data presentation system 160 located in the mobile terminaldevice 140 may be taken as an example to the description.

In some embodiments, the action data presentation system 160 may processdata and/or information related to action data presentation to performone or more functions described herein. In some embodiments, the actiondata presentation system 160 may obtain action data during the user'smotion, for example, an action signal during the user's motion collectedby the wearable apparatus 130, or data obtained after the action signalcollected during the user's motion by the wearable device 130 isprocessed by the processing device 110. In some embodiments, the actiondata presentation system 160 may send a control instruction to themobile terminal device 140 to control the display of the user interfaceof the mobile terminal device 140.

In some embodiments, the system 100 may also include a database. Thedatabase may store data (e.g., an initially preset threshold condition,etc.) and/or instructions (e.g., a feedback instruction). In someembodiments, the database may store data obtained from the wearableapparatus 130 and/or the mobile terminal device 140. In someembodiments, the database may store information and/or instructions forexecution or use by the processing device 110 to perform the exemplarymethods described in the present disclosure. In some embodiments, thedatabase may be connected with the network 120 to communicate with oneor more components of the system 100 (e.g., the processing device 110,the wearable apparatus 130, the mobile terminal device 140, etc.). Oneor more components of the system 100 may access data or instructionsstored in the database through the network 120. In some embodiments, thedatabase may connect or communicate directly with one or more componentsin the system 100. In some embodiments, the database may be portion ofthe processing device 110.

FIG. 2 illustrates a schematic diagram of exemplary hardware and/orsoftware components of a wearable apparatus according to someembodiments of the present disclosure. As shown in FIG. 2 , the wearableapparatus 130 may include an acquisition module 210, a processing module220 (also referred to as a processor), a control module 230 (alsoreferred to as a main controller, an MCU, a controller), a communicationmodule 240, a power supply module 250, and an input/output module 260.

The acquisition module 210 may be configured to obtain an action signalduring a motion of a user. In some embodiments, the acquisition module210 may include a sensor unit, and the sensor unit may be configured toobtain one or more action signals during the user's motion. In someembodiments, the sensor unit may include, but is not limited to one ormore of an EMG sensor, a posture sensor, an electrocardiogram sensor, arespiration sensor, a temperature sensor, a humidity sensor, an inertialsensor, a blood oxygen saturation sensor, a Hall sensor, anelectrodermal sensor, a rotation sensor, or the like. In someembodiments, the action signal may include one or more of an EMG signal,a posture signal, an electrocardiogram signal, a respiratory rate, atemperature signal, a humidity signal, or the like. The sensor unit maybe placed in different positions of the wearable apparatus 130 accordingto the type of the action signal to be obtained. For example, in someembodiments, the EMG sensor (also referred to as an electrode element)may be disposed at a position of a human muscle, and the EMG sensor maybe configured to collect an EMG signal during the user's motion. The EMGsignal and the corresponding feature information (e.g., frequencyinformation, amplitude information, etc.) thereof may reflect a state ofthe muscle during the user's motion. The gesture sensor may be set atdifferent positions of the human body (e.g., positions corresponding tothe trunk, limbs, and joints in the wearable apparatus 130), and thegesture sensor may be configured to collect the gesture signal duringthe user's motion. The gesture signal and the corresponding featureinformation (e.g., an angular velocity direction, an angular velocityvalue, an angular velocity acceleration value, an angle, displacementinformation, a stress, etc.) thereof may reflect the gesture of theuser's motion. The ECG sensor may be arranged at a position around thechest of the human body, and the ECG sensor may be configured to collectthe ECG data during the user's motion. The respiration sensor may bearranged at a position around the chest of the human body, and therespiration sensor may be configured to collect respiration data (e.g.,a respiration frequency, a respiration amplitude, etc.) during theuser's motion. The temperature sensor may be configured to collecttemperature data (e.g., a body surface temperature) during the user'smotion. The humidity sensor may be configured to collect humidity dataof the external environment during the user's motion.

The processing module 220 may process data from the acquisition module210, the control module 230, the communication module 240, the powersupply module 250, and/or the input/output module 260. For example, theprocessing module 220 may process the action signal during the user'smotion from the acquisition module 210. In some embodiments, theprocessing module 220 may preprocess the action signal (e.g., an EMGsignal, a gesture signal) obtained by the acquisition module 210. Forexample, the processing module 220 may perform segmentation processingon the EMG signal or the gesture signal during the user's motion. Asanother example, the processing module 220 may perform preprocessing(e.g., filtering processing, signal correction processing) on the EMGsignal during the user's motion to improve the quality of the EMGsignal. As further another example, the processing module 220 maydetermine feature information corresponding to the gesture signal basedon the gesture signal during the user's motion. In some embodiments, theprocessing module 220 may process an instruction or an operation fromthe input/output module 260. In some embodiments, the processed data maybe stored in a memory or a hard disk. In some embodiments, theprocessing module 220 may transmit the processed data to one or morecomponents of the motion monitoring system 100 via the communicationmodule 240 or the network 120. For example, the processing module 220may send a monitoring result of the user's motion to the control module230, and the control module 230 may execute subsequent operations orinstructions according to an action determined result.

The control module 230 may be connected with other modules of thewearable apparatus 130. In some embodiments, the control module 230 maycontrol operating states of other modules of the wearable apparatus 130.For example, the control module 230 may control a power supply state(e.g., a normal mode, a power saving mode), a power supply time, etc.,of the power supply module 250. As another example, the control module230 may control the input/output module 260 according to the user'saction determined result, so as to control the mobile terminal device140 to send a feedback result of the user's motion to the user. If thereis a problem with the action (e.g., the action does not meet thestandard) of the user during the user's motion, the control module 230may control the input/output module 260, so as to control the mobileterminal device 140 to give feedback to the user, so that the user mayknow his/her own motion state in real-time and adjust the action. Insome embodiments, the control module 230 may also control one or moresensors of the acquisition module 210 or other modules to providefeedback to the human body. For example, if a certain muscle exerts toomuch force during the user's motion, the control module 230 may controlan electrode module at the position of the muscle to electricallystimulate the user to prompt the user to adjust the action in time.

In some embodiments, the communication module 240 may be configured toexchange information or data. In some embodiments, the communicationmodule 240 may be configured for communication between components of thewearable apparatus 130. For example, the acquisition module 210 may senda user action signal (e.g., an EMG signal, a gesture signal, etc.) tothe communication module 240, and the communication module 240 may sendthe action signal to the processing module 220. In some embodiments, thecommunication module 240 may also be configured for communicationbetween the wearable apparatus 130 and other components in the system100. For example, the communication module 240 may send stateinformation (e.g., a switch state) of the wearable apparatus 130 to theprocessing device 110, and the processing device 110 may monitor thewearable apparatus 130 based on the state information. The communicationmodule 240 may adopt wired, wireless, and wired/wireless hybridtechnologies.

In some embodiments, the power supply module 250 may provide power toother components in the system 100.

The input/output module 260 may obtain, transmit, and send a signal. Theinput/output module 260 may interface or communicate with othercomponents in the system 100. Other components in the motion monitoringsystem 100 may be connected or communicated through the input/outputmodule 260.

It should be noted that the above descriptions of the system 100 and themodules thereof are merely for the convenience of descriptions, andcannot limit one or more embodiments of the present disclosure to thescope of the illustrated embodiments. It can be understood that forthose skilled in the art, after understanding the principle of thesystem, it is possible to arbitrarily combine the various modules, orform a subsystem to connect with other modules, or omit one or moremodules. For example, the acquisition module 210 and the processingmodule 220 may be integrated into one module which may have thefunctions of obtaining and processing the user action signal. As anotherexample, the processing module 220 may not be provided in the wearableapparatus 130 but integrated in the processing device 110. Suchmodifications are within the protection scope of one or more embodimentsof the present disclosure.

FIG. 3 illustrates a schematic diagram of exemplary hardware and/orsoftware components of a computing device 300 according to someembodiments of the present disclosure. In some embodiments, theprocessing device 110 and/or the mobile terminal device 140 may beimplemented on the computing device 300. In some embodiments, the actiondata presentation system 160 may be implemented on the computing device300. As shown in FIG. 3 , the computing device 300 may include aninternal communication bus 310, a processor 320, a read-only memory 330,a random-access memory 340, a communication port 350, an input/outputinterface 360, a hard disk 370, and a user interface 380.

The internal communication bus 310 may enable data communication amongthe various components of the computing device 300. For example, theprocessor 320 may send data to memory or other hardware components suchas the input/output interface 360 through the internal communication bus310.

The processor 320 may perform a computing instruction (a program code)and perform the functions of the motion monitoring system 100 describedherein. The computing instruction may include a program, an object, acomponent, a data structure, a procedure, a module, and a function (thefunction refer to the specific function described in the disclosure).For example, the processor 320 may process an action signal (e.g., anEMG signal, a posture signal) obtained from the wearable device 130or/and the mobile terminal device 140 of the motion monitoring system100 during the user's motion, and monitor the action of the useraccording to the action signal during the user's motion. Forillustration only, the computing device 300 in FIG. 3 only depicts oneprocessor, but it should be noted that the computing device 300 in thepresent disclosure may also include a plurality of processors.

The memory (e.g., a read-only memory (ROM) 330, a random-access memory(RAM) 340, a hard disk 370, etc.) of the computing device 300 may storedata/information obtained from any other component of the motionmonitoring system 100. In some embodiments, the memory of computingdevice 300 may be located in wearable apparatus 130 as well as inprocessing device 110.

The input/output interface 360 may be configured to input or output asignal, data, or information. In some embodiments, the input/outputinterface 360 may allow a user to interact with the motion monitoringsystem 100.

The hard disk 370 may be configured to store information and datagenerated by or received from the processing device 110. For example,the hard disk 370 may store user confirmation information of the user.In some embodiments, the hard disk 370 may be provided in the processingdevice 110 or in the wearable apparatus 130. The user interface 380 mayenable interaction and exchange of information between the computingdevice 300 and the user. In some embodiments, the user interface 380 maybe configured to present motion recordings generated by the motionmonitoring system 100 to the user. In some embodiments, the userinterface 380 may include a physical display, such as a display with aspeaker, an LCD display, an LED display, an OLED display, an electronicink display (E-Ink), or the like.

For example, the wearable apparatus 130 in the system 100 may adopt anystructure. For example, the wearable apparatus 130 may adopt a structureof the wearable apparatus 400 shown in FIG. 4 . In order to describe thewearable apparatus 130, the wearable apparatus 400 in FIG. 4 may takethe clothes above as an example. As shown in FIG. 4 , the wearableapparatus 400 may include an upper garment 410. The upper garment 410may include an upper garment base 4110, one or more upper garmentprocessing modules 4120, one or more upper garment feedback modules4130, one or more upper garment acquisition modules 4140, or the like.The upper garment base 4110 may refer to a clothing worn on the upperbody of the human body. In some embodiments, the upper garment base 4110may include a short-sleeved T-shirt, a long-sleeved T-shirt, a shirt, ajacket, or the like. The one or more upper garment processing modules4120 and the one or more upper garment acquisition modules 4140 may belocated on the upper garment base 4110 in areas that fit with differentportions of the human body. The one or more upper garment feedbackmodules 4130 may be located at any position on the upper garment base4110, and the one or more upper garment feedback modules 4130 may beconfigured to feed back motion state information of the user's upperbody. Exemplary feedback techniques may include, but are not limited to,a voice prompt, a text prompt, a pressure prompt, an electricalstimulation, or the like. In some embodiments, the one or more the uppergarment acquisition modules 4140 may include, but are not limited to oneor more of a posture sensor, an ECG sensor, an EMG sensor, a temperaturesensor, a humidity sensor, an acid-base sensor, a sound wave transducer,or the like. The sensors in the upper garment acquisition module 4140may be placed at different positions on the user's body according todifferent signals to be measured. For example, when the posture sensoris configured to obtain a posture signal during the user's motion, theposture sensor may be placed in the positions of the upper garment base4110 corresponding to the torso, arms, and joints. As another example,when the EMG sensor is configured to obtain an EMG signal during theuser's motion, the EMG sensor may be located near the muscles to bemeasured by the user. In some embodiments, the posture sensor mayinclude, but is not limited to, an acceleration triaxial sensor, anangular velocity triaxial sensor, a magnetic force sensor, etc., or anycombination thereof. For example, the posture sensor may include anacceleration triaxial sensor and an angular velocity triaxial sensor. Insome embodiments, the posture sensor may also include a strain gaugesensor. The strain gauge sensor may refer to a sensor that may be basedon the strain generated by the force and deformation of an object to bemeasured. In some embodiments, the strain gauge sensor may include, butis not limited to, one or more of a strain gauge load cell, a straingauge pressure sensor, a strain gauge torque sensor, a strain gaugedisplacement sensor, a strain gauge acceleration sensor, or the like.For example, the strain gauge sensor may be set at the user's jointposition, and by measuring the resistance of the strain gauge sensorthat changes with the stretched length, a bending angle and a bendingdirection of the user's joint may be obtained. It should be noted thatin addition to the above-mentioned upper garment base 4110, the upperprocessing module 4120, the upper garment feedback module 4130, and theupper garment acquisition module 4140, the upper garment 410 may alsoinclude other modules, such as a power supply module, a communicationmodule, an input/output module, etc. The upper garment processing module4120 may be similar to the processing module 220 in FIG. 2 , and theupper garment acquisition module 4140 may be similar to the acquisitionmodule 210 in FIG. 2 . For more descriptions of the various modules ofthe upper garment 410, please refer to FIG. 2 and the relateddescriptions thereof in the present disclosure, which are not repeatedhere.

FIG. 5 is a flowchart illustrating an exemplary process for determininga target action according to some embodiments of the present disclosure.In some embodiments, the process 500 may be implemented by the system100. For example, the memory in the processing device 110 and/or themobile device 140 may store one or more sets of action analysis andidentification instructions. The set of instructions may include aplurality of instructions. The processor of the processing device 110and/or the mobile device 140 may read and execute the plurality ofinstructions in the set of instructions at runtime, and execute theprocess 500 under the guidance of the plurality of instructions. Theprocess 500 may be completed in real-time, or each operation may becompleted in different time periods. The process 500 may include thefollowing operations.

In operation 510, user action data during a user's motion may beobtained. The user action data may correspond to an unknown user action.

The processing device 110 and/or the mobile device 140 may measure theabove-mentioned action data from a plurality of measurement positions onthe user, for example, may obtain raw data during the user's motion.Specifically, the processing device 110 and/or the mobile device 140 maybe communicatively connected with the wearable apparatus 130 directly orthrough the network 120. The wearable apparatus 130 may have a pluralityof sensors. When the user wears the wearable apparatus 130, theplurality of sensors may be attached to a plurality of positions on theuser's body. Therefore, the processing device 110 and/or the mobiledevice 140 may obtain measurement results of the plurality of sensorsattached to the user at the plurality of measurement positions throughthe corresponding acquisition modules to obtain the action data of theuser.

The user action data may refer to action data generated based on humanbody parameter information during the user's motion. In someembodiments, the human body parameter information may include, but isnot limited to, one or more of an EMG signal, a posture signal, anelectrocardiographic signal, a temperature signal, a humidity signal, ablood oxygen concentration, a respiratory rate, or the like.

Since a user action is a coordination result of a plurality of musclesand joints, correspondingly, the user action data may also include datacollected by multiple sensors at M positions on the user's body, whereinM is an integer greater than 1. In some embodiments, the data collectedby each individual sensor may be considered as a piece of actionsub-data. For example, in some embodiments, the plurality of sensors inthe wearable apparatus 130 may obtain signals of a plurality of bodyparts during the user's motion. The combination of the gesture signalsof the plurality of body parts may reflect the relative motion situationbetween different parts of the human body. For example, an EMG sensor inthe wearable apparatus 130 may collect an EMG signal during the user'smotion, a posture sensor in the wearable apparatus 130 may collect aposture signal during the user's motion, and an angle sensor and anangular velocity sensor in the wearable device 130 may collect an angleand an angular velocity of each joint during the user's motion. Thesignal of each sensor of the above-mentioned sensors is recorded as apiece of action sub-data. All the action sub-data may be combined toform the action data.

For example, when a person performs an arm curling action, thecorresponding action data may include angle data and angular velocitydata of the upper arm, angle data, and angular velocity data between theupper arm and the forearm, EMG data of biceps brachii muscle, EMG dataof deltoid muscle, EMG data of trapezius muscle, and data of back musclegroup, etc., measured by the wearable apparatus 130. As another example,when the user performs seated chest clamping, the EMG sensor in thewearable apparatus 130 corresponding to the position of the human body'spectoralis muscle, latissimus dorsi, etc., may collect the EMG signal ofthe user's corresponding muscle position. As further another example,when the user performs a squat, the EMG sensor in the wearable apparatus130 corresponding to the position of the human gluteus maximus,quadricep, etc., may collect the EMG signal of the user's correspondingmuscle position, and the nearby angle sensor and the angular velocitysensor may collect the angle and the angular velocity between the thighand the calf. The data collected by each single sensor may be regardedas a piece of action sub-data. Therefore, the user action data of thearm curling action may also include a plurality of user action sub-data,respectively corresponding to the data collected at the plurality ofpositions of the user's body when the user performs the arm curlingaction.

When the user wears the wearable apparatus 130, the wearable apparatus130 may start to measure the user's motion at any time, and the user maystart to exercise or start to rest at any time. Thus, the processingdevice 110 and/or the mobile device 140 do not know whether the wearableapparatus 130 collects a segment of user's random action data or theaforementioned fitness action, nor does the processing device 110 and/orthe mobile device 140 know when did the fitness action start in thecollected data. Therefore, what the wearable apparatus 130 collects isaction data whose content is unknown. That is to say, the system 100 isin an unknown state about what kind (that is, whether the user actionincludes several known target actions, and a start time of each targetaction) of motion the user is doing and when the user starts exercising.Therefore, the action data also has no identifier indicating the contentof the action. Merely for the convenience of description, the presentdisclosure may take whether the action data includes an actionidentifier as an example to describe whether the content of the actiondata is known.

In operation 520, whether the user action includes a reference action(i.e., a target action) corresponding to one or more sets of candidatereference action data may be identified based on the one or more sets ofcandidate reference action data whose content is known by performingaction identification on the user action data.

The target action may refer to a specific action in the actual actionsperformed by the user, such as biceps curling, bench press, squat, kick,push-up, deadlift, abdominal crunches, etc. The reference action mayrefer to a standard action with an action content marked on the specificaction performed by a reference person (such as a coach, etc.). In someembodiments, the target action identification may be performedimmediately or within a preset time after the user action data isobtained. The preset time may be a very short time, for example, withinone hour or 0.1, 0.5 seconds. The target action identification may alsobe performed in real time with the acquisition of the user action data.For example, when the user is performing an action, the processor of theprocessing device 110 and/or the mobile device 140 may obtain the useraction data while simultaneously stretching or compressing the useraction data to a different scale and then comparing it to the referenceaction data, thereby simultaneously performing the action identificationon the user action data. The method of comparison described above may bedescribed in other parts of the present disclosure.

The reference action data may include a plurality of sets of referenceaction data measured by a plurality of sensors. When collecting thereference action data, similar to the collection of the user actiondata, the plurality of sensors may also be attached to the M measurementpositions of the reference person, for example, the reference person maywear the wearable apparatus 130. Then the reference person may performthe specific action, and the data acquisition device (e.g., theprocessing device 110 and/or the mobile device 140, or other devices)may receive the corresponding action data through the correspondingacquisition module in the wearable apparatus 130. Therefore, thecandidate reference action data may be action data of a reference actionwhose content is known and measured from the M measurement positions onthe reference person, such as reference action data marked with anaction content. For example, the reference action data of the upper armcurling may include data collected from the same plurality of positionson the reference person. The reference person may be a person who isused as a model to collect the reference action data, such as a fitnesscoach. For the data collected by each sensor, the reference action dataof the arm curling may include M pieces of reference action sub-dataeach of which corresponds to the data collected at one of the Mpositions of the body when the reference person performs the arm curlingaction. In some embodiments, the M measurement positions on thereference person may have the one-to-one correspondence relationshipwith the M measurement positions on the user when the user action datais collected. Correspondingly, the M pieces of user action sub-data ofthe user action data may have the one-to-one correspondence relationshipwith the M pieces of reference action sub-data.

The reference action data may also be generated in other ways. Forexample, action data of a virtual person may be obtained throughcomputer modeling, and the reference action data may be obtained byartificially fitting the virtual person in the video through artificialintelligence techniques (for example, artificially fitting Mr. Olympia'saction demonstration video). As long as certain action data maystandardly represent a certain action, and an action content of thecertain action data is known, the certain action data may be used as thereference action data in the present disclosure. As mentioned above,merely for the convenience of description, the present disclosure maytake the action data marked with the content as an example to describethe action data with known content.

In operation 520, the processing device 110 and/or the mobile device 140may access a reference action database. The reference action databasemay include the plurality of sets of reference action data. Theplurality of sets of reference action data may correspond to a pluralityof reference actions. For example, each set of the reference action datamay correspond to one reference action, or each set of the referenceaction data may correspond to a plurality of reference actions, or eachreference action may correspond to the plurality of sets of referenceaction data.

When performing target action identification on the user action data,the processing device 110 and/or the mobile device 140 may identify theuser action data by sequentially comparing the user action data witheach of the plurality of sets of reference action data in the referenceaction database through a two-level screening operation. The two-levelscreening operation may screen the plurality of sets of reference actiondata through two different screening operations, and finally determinewhich reference action the user's action includes. For example, thetwo-level screening operation may include a combination of a differencedegree-based screening operation and a probability-based screeningoperation.

Specifically, in the first level screening, the processing device 110and/or the mobile device 140 may select a set of candidate referenceaction data as first-level candidate reference action data, and then usethe selected candidate reference action data to determine a differencedegree of the action with the user action data to determine whether adifference in data values between the user action data and thefirst-level candidate reference action data is sufficiently small. Ifthe difference degree between the user action data and a certainfirst-level candidate reference action data is less than a preset value,the first-level candidate reference action data may be promoted to asecond-level candidate reference action data. Each second-levelcandidate reference action data corresponds to one or more promotedreference actions, that is, the second-level reference action.

The second level screening may be a probabilistic screening. In thesecond level screening, the processing device 110 and/or the mobiledevice 140 may determine the probability that the user action includesthe promoted reference action (second-level reference action), and thendetermine whether the user action includes the second-level referenceaction. Whether the user action includes the target action correspondingto target reference action data may be determined based on the result ofthe second level screening. Specific operations are described asfollows.

In operation 521, each set of reference action data may be selected inturn from the plurality of sets of reference action data as thefirst-level candidate reference action data.

In operation 522, a difference degree between the first-level candidatereference action data and the user action data may be determined.

In operation 523, whether a value of the difference degree is less thana first preset value may be judged. If the difference degree value isgreater than or equal to the first preset value, the overall differencebetween the user action and the first-level candidate reference actiondata may be considered relatively large. Then operation 521 may beperformed, that is, the next set of reference data may be selected fromthe plurality of sets of reference action data in the reference actiondatabase as the first-level candidate reference action data and the nextset of reference data may be compared with the user action data again atthe data value level. If the difference degree value of the next set ofreference data is less than the first preset value, the overalldifference between the user action and the first-level candidatereference action data may be considered small. Then operation 524 may beperformed, that is, the first-level candidate reference action data maybe determined as the second-level candidate reference action data, andthen the next-level target action data identification may be performed.

In operation 525, a distance between the user action data and each ofthe multiple sets of the second-level candidate reference action datamay be determined.

In operation 526, a probability that the user action data includes thetarget action corresponding to the second-level candidate referenceaction data may be determined based on each distance.

In operation 527, whether the maximum value among the values of theprobabilities is greater than a second preset value is judged. If themaximum value is not greater than the second preset value, operation 529may be performed, that is, it is determined that the user action doesnot include the reference action corresponding to the second-levelcandidate reference action data. If the maximum value is greater thanthe second preset value, operation 528 may be performed, that is, it isdetermined that the user action includes the reference actioncorresponding to the second-level candidate reference action data withthe highest probability value, and the reference action is the targetaction.

The basis for selecting the first-level candidate reference action datamay be random and in sequence, or may be selected according to a certainrule, which is not limited in the present disclosure. For example, allreference action data may be numbered in the reference action databasein advance, and then the processing device 110 and/or the mobile device140 may select the reference action data item by item as the first-levelcandidate reference action data according to the number.

When comparing the first-level candidate reference action data with theuser action data, a sliding window comparison manner may be adopted. Forexample, the processing device 110 and/or the mobile device 140 mayslide a sliding window over the user action data along the time axis andselect a segment of the user action data within the sliding window.Since the M pieces of user action sub-data in the user action data arecollected in parallel, the sliding window may act on each piece of useraction sub-data at the same time, and slide over each piece of useraction sub-data in parallel. The sliding window may correspond to apreset time interval (such as 0.1 seconds, 0.5 seconds, 1 second, etc.).Therefore, for the M pieces of user action sub-data, the sliding windowmay include M data segments of the user action data collected in thepreset time interval. The processing device 110 and/or the mobile device140 may respectively compare the M user action data segments with someor all of the data of the M pieces of reference action sub-datacorresponding to the first-level candidate reference action data toobtain one or more comparison sub-results and then determine acomprehensive difference degree by weighting and summing the one or morecomparison sub-results. The comprehensive difference degree indicatesthe difference between the user action data and the reference actiondata. The smaller the value of the comprehensive difference degree is,the smaller the difference is, which indicates that the closer the useraction data segment is to the reference action data, the closer the useraction corresponding to the user action data segment is to the referenceaction, and the processing device 110 and/or the mobile device 140 maydetermine that the user action data includes the reference action. Forexample, when a user performs biceps curling during the user's motion,the user action data may include a corresponding user action datasegment. When the processing device 110 and/or the mobile device 140compares the user action data segment corresponding to the bicepscurling with the reference action data corresponding to the bicepscurling, the comprehensive difference degree value may be very small. Onthe other hand, the smaller the value of the comprehensive differencedegree may indicate that the closer the position of the user action datasegment in the user action data is to the position of the target actionin the user action data, that is, the user action corresponding to theuser action data segment is closer in time to the moment when the userperforms the target action.

Specifically, the processing device 110 and/or the mobile device 140 mayuse a sliding window with a preset width to slide over the user actiondata along the time axis with a preset step size, and select a useraction data segment within the sliding window each time. For example,the processing device 110 and/or the mobile device 140 may sequentiallyselect a segment of continuous data with a preset data length on theuser action data with the preset step size. Considering that the speedof the user performing the target action may be different from the speedof the standard action performed by the reference person, the slidingwindow length may be negatively correlated with the use action speed tooffset the difference. That is, when the user action speed is faster,the taken sliding window length is shorter, and when the user actionspeed is slower, the taken sliding window length is longer.

The preset step size may be a constant value. Since the value of thecomprehensive difference degree also denotes a temporal distance betweenthe user action corresponding to the current user action data segmentand the target action, the preset step size may also be adjusted basedon the value of the comprehensive difference degree. For example, inorder to increase the efficiency of identifying the target action, thepreset step size may be positively correlated with the magnitude of thevalue of the comprehensive difference degree at the previous moment. Thepositive correlation may indicate that the preset set size isproportional to the value of the comprehensive difference degree at theprevious moment, or may select the step size of the current moment in acertain step manner based on the value of the comprehensive differencedegree at the previous moment, or may be that the step size of thecurrent moment is greater than the step size of the previous moment by aconstant, etc., which is not limited here. The preset step size may alsobe positively correlated with the variation trend of the value of thecomprehensive difference degree. For example, if the difference betweenthe comprehensive difference degree value at the current moment and thecomprehensive difference degree value at the previous moment is greaterthan 0, i.e., the variation trend of the comprehensive difference degreevalue is increasing, which means that the user action corresponding tothe current user action data segment is getting farther and farther awayfrom the target action in time. At this time, the processing device 110and/or the mobile device 140 may increase the step size. If thedifference between the comprehensive difference degree value at thecurrent moment and the comprehensive difference degree value at theprevious moment is less than 0, which means that the user actioncorresponding to the current user action data segment is getting closerand closer to the target action in time. At this time, the processingdevice 110 and/or the mobile device 140 may reduce the step size. If thedifference between the comprehensive difference degree value at thecurrent moment and the comprehensive difference degree value at theprevious moment is equal to 0, the step size may be kept unchanged.

Since the width of the sliding window is preset, the length of the datasegment intercepted from the user action data may also be preset.Therefore, the user action data segment may correspond to the entirefirst-level candidate reference action data. The user action datasegment may also correspond to a portion of the first-level candidatereference action data. In some embodiments, the reference action datamay include one or more segments of action identification data. Theaction identification data may be action data (e.g., angular velocitydata, velocity data, etc.) corresponding to at least a portion of thecharacteristic action of the reference action, which is essentiallyconfigured to represent the characteristic action. The characteristicaction may be unique to the reference action. The reference action maybe determined through the portion of the characteristic action, or theentire data may be determined as the reference action data through theaction identification data, so that when a data segment similar to theaction identification data appears in the user action data, the useraction may be recognized as including the corresponding target action.Meanwhile, the action identification data may only exist on a portion ofthe reference action sub-data of the reference action data, or theaction identification data may exist in each reference action sub-data.The action identification data existing on the reference action sub-datamay be referred to as action identification sub-data.

For the sliding window at the current moment, the processing device 110and/or the mobile device 140 may compare the M user action data segmentswith the corresponding M pieces of action identification sub-datarespectively to obtain the corresponding M difference degrees. The Mdifference degrees may be weighted and summed to obtain thecomprehensive difference degree. If the comprehensive difference degreeis less than the first preset value, the first-level candidate referenceaction data having passed the first level screening may be determined,and the first-level candidate reference action data may be selected asthe second-level candidate reference action data. If the value of thecomprehensive difference degree is greater than the first preset value,the sliding window may slide to the next user action data segment withthe preset step size, and then the comparison may be repeated until thecomprehensive difference degree value is less than the first presetvalue or the sliding window slides to the end of the user action data.

Specifically, when comparing a certain segment of action identificationsub-data of a certain piece of first-level candidate reference actiondata with its corresponding user action sub-data, the followingoperations may be performed.

For a certain segment of action identification sub-data in thefirst-level candidate reference action data, the data istwo-dimensional. For example, the first-level candidate reference actiondata of the arm curling may include action identification sub-data atdifferent time points for the bending angle of the forearm relative tothe upper arm. The action identification sub-data may include aplurality of angle values and a plurality of corresponding time points,so the action identification sub-data is two-dimensional data. For suchsingle parameter action identification sub-data, when a segment ofaction identification sub-data is included in one piece of thefirst-level candidate reference action data, the specific process forobtaining the comprehensive difference degree is as follows. FIG. 7Ashows an exemplary segment of action identification data in referenceaction data and a curve of a segment of the user action sub-datacollected by the sliding window in the user action data on the time axisaccording to some embodiments of the present disclosure. The actionidentification sub-data in the reference action data may include aplurality of pieces of data {a_(j)}={a₁, a₂, a₃, . . . , a_(n)}, whereinn is a_(n) integer greater than 1, each piece of the data corresponds toa timestamp, and as j increases, the time point corresponding to thetimestamp of each data a_(j) increases sequentially. That is, the datapoints in the vector {a_(j)} may be arranged in chronological order. Theuser action sub-data segment may include a plurality of pieces of data{b_(i)}={b₁, b₂, a₃, . . . , b_(m)}, wherein m is a_(n) integer greaterthan 1, each piece of the data corresponds to a timestamp, and as iincreases, the time point corresponding to the timestamp of each data biincreases sequentially. That is, the data points in the vector {b_(i)} amay be arranged in chronological order. Generally speaking, the timelength corresponding to the sliding window is less than the time lengthcorresponding to the action identification sub-data, and the amount ofdata corresponding to the sliding window is less than the amount of datacorresponding to the automatic identification data, that is, m<n.

Assuming that sampling frequencies and action speeds of the referenceaction data and the user action data are the same, for the same timeinterval, the quantity of data points of the reference action data andthe quantity of data points of the user action data may be the same.Thus, the user action sub-data segment {b_(i)} may correspond to thedata of the same length in the action identification sub-data {a_(j)}.That is, each data point in {b_(i)} may correspond to a data point in{a_(j)}. When determining the difference degree between {b_(i)} and{a_(j)}, {b_(i)} only needs to be drawn along the time axis t, adistance of one data point can be slid each time, and once differencedegree determination between the data point and the corresponding datapoint in {a_(j)} may be determined every time. However, considering thatthe sampling frequencies and/or the action speeds of the referenceaction data and the user action data are inconsistent, there is noone-to-one correspondence between the data points of the user action andthe data points of the reference action. At this time, an action-timerelationship of {b_(i)} needs to be adjusted according to various timescales to make the adjusted action-time relationship consistent with theaction-time relationship of the reference action data. For example, ifthe sampling frequencies of the reference action data and the useraction data are the same but the action speeds are inconsistent, thetime taken by the user to perform an action is different from the timetaken by the corresponding reference action. For example, if thehardware sampling frequency is 100 data points per second, a speed of ajoint in the reference action changes from 0° to 90° in 1 second, and aspeed of the user action may only change from 0° to 45° in 1 second, for100 data points, the reference action data corresponds to an anglechange of 90° and the user action data corresponds to an angle change of45°. Thus, the time span of {b_(i)} needs to be adjusted according tovarious time scales, i.e., stretching or compressing {b_(i)} accordingto different scales, and then discriminating the difference degreebetween the processed data point and the corresponding data point in{a_(j)} once for each scale, until all the difference degreescorresponding to all scales are determined. The specific operations maybe described as follows.

Firstly, the processing device 110 and/or the mobile device 140 maycalculate a distance D_(ji) between any point b_(i) in the user actionsub-data and any point a_(j) in the action identification sub-data byplacing the user action sub-data segment {b_(i)} and the actionidentification sub-data {a_(j)} on the same time axis, and determine them×n distance matrix D_(m×n) as shown in FIG. 7B. Each element in thedistance matrix denotes a distance from the i-th (i≤m) point of the useraction sub-data segment to the j-th (i≤n) point of the actionidentification sub-data in the first-level candidate reference actiondata. Taking a fitness action such as biceps curling as an example ofthe user action, the user action sub-data {bi} may include anglesbetween the upper arm and the forearm, and the corresponding actionidentification sub-data {aj} in the reference data may also includeangles between the upper arm and the forearm. Thus, the distance D_(i);may indicate a difference D_(ji)=|a_(j)−b_(i)| between the angle betweenthe upper arm and the forearm of the user and the angle represented bythe action identification sub-data. The distance D₅₆ between a₅ and b₆and the distance D₅₃ between a₅ and b₃ are shown in FIG. 7A. Of course,the distance may also be a distance defined in other ways. For example,a distance between any point b_(i) in the user action sub-data and anypoint a_(j) in the action identification sub-data may be a Euclideandistance, a Manhattan distance, a P-parametric distance, a cosinedistance, a Chebyshev distance, a Marxian distance, an edit distance, aJaccard distance, or any other correlation distance. In this way, thedistance matrix D_(m×n) includes point-to-point distances between allpoints in the user action sub-data segment {b_(i)} on all scales and allpoints in the action identification sub-data {a_(j)}.

Secondly, the processing device 110 and/or the mobile device 140 maydetermine the shortest distance path P_(min) in the distance matrixD_(m×n), i.e., the smallest regularization cost. The distance path maybe expressed by the following vector, P={p_(k)}={p₁, p₂, . . . , p_(l)},which is a sequence composed of some elements of the distance matrixD_(m×n), wherein l denotes a count of elements in the distance path P.The distance path P may include a plurality of numbers, each of which isan element (i.e., a distance) in D_(m×n). Any two adjacent numbers aretwo adjacent elements of the distance matrix D_(m×n), and a position ofthe next number of the sequence in the distance matrix D_(m×n) is to theright, below, or below the right of a corresponding position of theprevious number of the sequence in the distance matrix D_(m×n). Sincethe time corresponding to the user action sub-data segment in thesliding window is shorter than the time corresponding to the actionidentification sub-data, two ends of the shortest distance path P maycorrespond to the first data point b₁ and the last data point b_(m) ofthe {b_(i)}, that is, the first number in the sequence is p₁=D_(1x), andthe last number is p_(i)=D_(my), wherein x<n, y<n, x and y denote thepositions of the corresponding data points in {a_(j)}, respectively. Theshortest distance path P_(min) is the one with the smallest sum of allelements among all the paths satisfying the above conditions, that is,P_(min)=P|_(min(Σ) _(k=1) _(l) _(p) _(k) ₎={p₁, p₂, p₃, . . . ,p_(l)}|_(min(Σ) _(k=1) _(l) _(p) _(k) ₎.

Considering that the sliding window may be set with different timelengths and different data sampling frequencies, the quantity of datapoints in the user action data segment in the time window may bedifferent. This may cause the value of the shortest distance path P tobe different, depending on the data sampling frequency and the length ofthe time window of the sensors. Taking these factors into consideration,the difference degree ƒ may be defined as ƒ=WP_(min) ^(T), whereinWP_(min) ^(T)=Σ_(i=1) ^(l)w_(i)p_(i) may be the weighted average of theelements of the shortest distance path P_(min), W={w₁, w₂, . . . w_(l)}may be a weight vector consisting of m elements with 1 row and lcolumns, and WP_(min) ^(T) is a scalar. For example, the differencedegree may be defined as an average distance, that is, all elements in Ware 1/l.

Through the above calculation, the processing device 110 and/or themobile device 140 may complete the operations of determining thedifference degree between each of the plurality of scaled data segmentsand the action identification sub-data, respectively, and determiningthe smallest difference degree among difference degrees between the datasegment and the action identification sub-data.

In some embodiments, in order to reduce the amount of computation anddetermine the adjustment scale of the user action sub-data, beforedetermining the difference degree of the user action sub-data segmentwithin the sliding window, the processing device 110 and/or the mobiledevice 140 may determine a target comparison data interval from theaction identification sub-data, and only compare the target comparisondata interval with the user action sub-data segment to obtain adifference degree. The target comparison data interval may include aplurality of action identification sub-data points. The targetcomparison data interval may be determined based on the followingrelationship between the user action data and the action identificationsub-data selected by the sliding window.

1) The beginning and end of the user action data segment selected by thesliding window exactly correspond to the beginning and end of the actionidentification sub-data, that is, the user action data segment isexactly the complete distribution of the action identification sub-data.In this case, the processing device 110 and/or the mobile device 140 mayfirst determine that the first data point and the last data point of theuser action data segment selected by the sliding window correspond tothe first data point and the last data point of the corresponding actionidentification sub-data, respectively. The target comparison datainterval covers the entire action identification sub-data. Theconstraint of the distance path P may include that: each number of thesequence is an element of D_(m×n), any two adjacent numbers are twoadjacent elements in the distance matrix D_(m×n), and the position ofthe next number of the sequence in the distance matrix D_(m×n) is to theright, below, or below the right of the corresponding position of theprevious number in the distance matrix D_(m×n). The two ends of theshortest distance path P may correspond to p₁=D₁₁ and p_(l)=D_(mn). Thatis, the shortest distance path P of the distance matrix is the shortestdistance path from the upper left corner to the lower right corner ofthe distance matrix.

2) The start point of the user action sub-data selected by the slidingwindow corresponds to the start point of the action identificationsub-data, and the end point of the user action sub-data selected by thesliding window corresponds to a certain data point in the actionidentification sub-data. That is, the user action sub-data correspondsto a segment in the action identification sub-data after scaling, andthis segment is located at the start position of the actionidentification sub-data. In this case, the processing device 110 and/orthe mobile device 140 may first determine that the first data point ofthe user action data segment selected by the sliding window correspondsto the first data point of the corresponding action identificationsub-data. Then, the target comparison data interval may cover the entireaction identification sub-data. The constraint of the distance path Pmay include that: each number of the sequence is an element in D_(m×n);any two adjacent numbers are two adjacent elements of the distancematrix D_(m×n), and the position of the next number of the sequence inthe distance matrix D_(m×n) is to the right, below, or below the rightof the corresponding position of the previous number in the distancematrix D_(m×n). Both ends of the shortest distance path P may correspondto p₁=D₁₁ and p_(l)=D_(my). That is, the shortest distance path P of thedistance matrix is a distance from the upper left corner of the distancematrix to a certain point of the last row along the lower rightdirection.

3) The start point of the user action sub-data selected by the slidingwindow corresponds to a certain data of the action identificationsub-data, and the end point of the user action sub-data selected by thesliding window corresponds to the end point of the action identificationsub-data. That is, after scaling, the user action sub-data may be asegment of data at the end position of the action identificationsub-data. In this case, the processing device 110 and/or the mobiledevice 140 may first determine that the last data point of the useraction data segment selected by the sliding window corresponds to thelast data point of the corresponding action identification sub-data.Then, the target comparison data interval may cover the entire actionidentification sub-data. The constraint of the distance path P mayinclude that: each number of the sequence is an element in D_(m×n), anytwo adjacent numbers are two adjacent elements of the distance matrixD_(m×n), and the position of the next number of the sequence in thedistance matrix D_(m×n) is to the right, below, or below the right ofthe corresponding position of the previous number in the distance matrixD_(m×n). Both ends of the shortest distance path P may correspond top₁=D_(1x) and p_(l)=D_(mn). That is, the shortest distance path P of thedistance matrix is the shortest distance path from a certain point inthe first row of the distance matrix to the lower right corner.

4) The start point and the end data point of the user action sub-dataselected by the sliding window correspond to the two intermediate datapoints of the action identification sub-data, respectively, rather thanthe first data point and the last data point thereof. That is, the startpoint of the user action sub-data selected by the sliding window may notbe the beginning of the action identification sub-data, and the endpoint of the user action sub-data selected by the sliding window mayalso be not the end of the action identification sub-data. Afterscaling, the user action sub-data may be a segment of data of the actionidentification sub-data, and this segment may be located at a certainposition in the middle of the action identification sub-data. The“intermediate data” of a segment of data may refer to data at anyposition except the start point and the end point of the data. In thiscase, the processing device 110 and/or the mobile device 140 may firstdetermine that the first data point and last data point of the useraction sub-data segment selected by the sliding window may not be thefirst data point and the last data point of the corresponding actionidentification sub-data. Then the target comparison data interval maycover the entire action identification sub-data except for the firstdata point and the last data point. The constraint of the distance pathP may include that: each number of the sequence is an element inD_(m×n), any two adjacent numbers are two adjacent elements of thedistance matrix D_(m×n), and the position of the next number of thesequence in the distance matrix D_(m×n) is to the right, below, or belowthe right of the corresponding position of the previous number in thedistance matrix D_(m×n). The two ends of the shortest distance path Pmay correspond to p_(l)=D_(1x) and p_(l)=D_(my), wherein, x∈(1, y],y∈[1, n). That is, the shortest distance path P of the distance matrixstarts from a certain point in the middle of the first row of thedistance matrix and extends to the lower right and ends at a certainpoint in the last row.

In some embodiments, the action identification sub-data may correspondto a start action or an end action of a reference action. At this time,when determining whether a certain point in the user action sub-data isthe start point or the end point of the action, it may be determined bythe change of the angular velocities before the point and after thepoint. For example, when a user action data point shows that the angularvelocity of the corresponding user action is 0, and the angular velocityof a point after the user action data point is not 0, it may be provedthat the user starts a certain fitness action from the user action datapoint, so it may be determined that the user action data point is thestart point of the user action. As another example, when the angularvelocity of a user's action point is 0, and the angular velocity of theprevious point is not 0, it may be proved that the user stopped doingthe action at this point, so it may be determined that the user's actionpoint is the end point of the action.

It should be understood that performing target action identification onthe user action data may include one of the following two cases. 1) Theaction identification sub-data may be compared with the user actionsub-data segment by segment to obtain a comprehensive difference degreevalue. When the value of the comprehensive difference degree is lessthan the first preset value, the processing device 110 and/or the mobiledevice 140 may determine that the first-level candidate reference actiondata is the second-level candidate reference action data. 2) The actionidentification data may be compared with the user action data segment bysegment to obtain a comprehensive difference degree value. When thevalue of the comprehensive difference degree is greater than the firstpreset value, the processing device 110 and/or the mobile device 140 mayslide the sliding window to the next data segment at the preset stepsize, and then repeat the comparison. The first preset value may be acriterion for determining whether the distance between the user actiondata and the action corresponding to the first-level candidate referenceaction data is sufficiently small. Therefore, when the value of thecomprehensive difference degree is less than the first preset value, thedistance between the user action data and the first-level candidatereference action data may be proved to be relatively small (that is, thesimilarity is very high), the user action data may be considered toinclude target action data corresponding to the first-level candidatereference action data. At this time, the first-level candidate referenceaction data may be determined to be the second-level candidate referenceaction data. When the comprehensive difference degree value is greaterthan the first preset value, the similarity between the user action dataand the first-level candidate reference action data may be proved to bevery low, and the user action data may be determined to not include thetarget action data corresponding to the reference action.

The above illustrates a method for determining the difference degreebetween a piece of user action sub-data and the corresponding actionidentification sub-data. FIG. 7C is a schematic diagram illustrating aprocess for determining a comprehensive difference degree when useraction data includes a plurality of pieces of user action sub-dataaccording to some embodiments of the present disclosure.

If action data of a certain action includes data for measuring Mparameters, and M is an integer greater than 1, the action data of theaction includes M pieces of parallel measurement sub-data. Therefore,the user action data may include M pieces of user action sub-data. Thefirst-level candidate reference action data may also include M pieces offirst-level candidate reference action sub-data. Each piece of thefirst-level candidate reference action sub-data corresponds to localaction data in the overall action data obtained by a parametermeasurement and includes at least a segment of independent actionidentification sub-data. All the action identification sub-data togetherconstitute the action identification data of the reference action.

When the first-level candidate reference action sub-data correspondingto a certain piece of user action sub-data includes a segment of actionidentification sub-data, the comparison sub-result between the useraction sub-data and the first-level candidate reference action sub-datamay be obtained based on the following operations. The processing device110 and/or the mobile device 140 may select a segment of the user actiondata from each of the M pieces of user action sub-data using a slidingwindow, and sequentially compare the data segments with thecorresponding M segments of action identification sub-data in the Mpieces of reference action sub-data to determine difference degrees, sothat the comparison sub-results may be obtained. Then, the M comparisonsub-results may be weighted and summed to obtain a comprehensivedifference degree, and whether the user action data includes the targetaction may be determined based on the comprehensive difference degreeand the first preset value. For each piece of the M pieces of useraction sub-data, the processing device 110 and/or the mobile device 140may obtain the difference degree between the user action sub-data andthe action identification sub-data based on the method described above.

Specifically, for each piece of the M pieces of user action sub-data,the processing device 110 and/or the mobile device 140 may collect auser action sub-data segment by a sliding window. The sliding windowscorresponding to the M pieces of the user action sub-data may be linkedor operated independently when sliding. Each sliding window may have thesame width, that is, the user action sub-data segment corresponding toeach sliding window may uniformly include d (d is an integer greaterthan 1) data points, and as described above, each data point of the ddata points corresponds to a timestamp. Of course, the widths ofdifferent sliding windows may also be different, and the amount of dataincluded in the user action sub-data segment in each sliding window mayalso be different. The processing device 110 and/or the mobile device140 may determine the overall distance between each data point of theuser action sub-data segment and each data point of the actionidentification sub-data according to the method described above. Theprocessing device 110 and/or the mobile device 140 may obtain theminimum regularization cost of the overall distance through the abovemethod, and then determine the difference degree between the user actionsub-data segment and the action identification sub-data. Since there areM pieces of user action sub-data, according to the above method, theprocessing device 110 and/or the mobile device 140 may obtain Mdifference degrees in total. Finally, the processing device 110 and/orthe mobile device 140 may perform weighted summation on the M comparisonsub-results to obtain a comprehensive difference degree.

In some embodiments of the present disclosure, after obtaining the valueof the comprehensive difference degree, the processing device 110 and/orthe mobile device 140 may directly determine whether the user actiondata includes the first-level candidate reference action data. Forexample, the processing device 110 and/or the mobile device 140 may seta first preset value for the action identification data, and when thevalue of the comprehensive difference degree is greater than the firstpreset value, the processing device 110 and/or the mobile device 140 mayslide each of the M sliding windows to the next data segment at thepreset step size, and then the comparison may be repeated. If thecomprehensive difference degree value is less than the first presetvalue, the processing device 110 and/or the mobile device 140 mayconsider that the set of user action sub-data may include thefirst-level candidate reference action data, thereby the above-mentionedloop may be ended.

When the first-level candidate reference action sub-data correspondingto a piece of user action sub-data includes a plurality of segments(e.g., p segments, p is an integer greater than 1) of actionidentification sub-data, a comparison result between the user actionsub-data and the first-level candidate reference action sub-data may beobtained based on the following method. For each piece of the M piecesof user action sub-data, the processing device 110 and/or the mobiledevice 140 may sample a user action sub-data segment from each piece ofthe user action sub-data through the sliding window. The sliding windowscorresponding to the M pieces of the user action sub-data may be linkedor operated independently when sliding. Each sliding window may have thesame width, that is, the user action sub-data segment corresponding toeach sliding window may uniformly include d (d is an integer greaterthan 1) data points, and as described above, each data point of the ddata points corresponds to a timestamp. Of course, the widths ofdifferent sliding windows may also be different, and the amount of dataincluded in the user action sub-data segment in each sliding window mayalso be different. The processing device 110 and/or the mobile device140 may calculate the p integral distances between the user actionsub-data segment and the p segments of action identification sub-datarespectively based on the method described above. The processing device110 and/or the mobile device 140 may determine the minimumregularization cost of the p overall distances by using the above methodas a comparison sub-result between the user action sub-data segment andthe action identification sub-data. Since there are M pieces of useraction sub-data, the processing device 110 and/or the mobile device 140may obtain M comparison sub-results in total based on the above method.Finally, the processing device 110 and/or the mobile device 140 mayperform weighted summation on the M comparison sub-results to obtain thecomprehensive difference degree value.

In some embodiments of the present disclosure, after obtaining the valueof the comprehensive difference degree, the processing device 110 and/orthe mobile device 140 may directly determine whether the set of useraction sub-data includes the first-level candidate reference actiondata. For example, the processing device 110 and/or the mobile device140 may set a first preset value for the action identification data, andwhen the comprehensive difference degree value is greater than the firstpreset value, the processing device 110 and/or the mobile device 140 mayslide each of the M sliding windows to the next data segment at thepreset step size, and then the comparison may be repeated. If the valueof the comprehensive difference degrees is less than the first presetvalue, the processing device 110 and/or the mobile device 140 maydetermine that the first-level candidate reference action data is thesecond-level candidate reference action data, thereby theabove-mentioned loop may be ended.

In some embodiments, after determining the second-level candidatereference action data, the processing device 110 and/or the mobiledevice 140 may further confirm whether the user action includes thereference action corresponding to the second-level candidate referenceaction data.

In some embodiments of the present disclosure, a second preset value mayalso be set, and the second preset value may be a preset value relatedto the probability. Assuming that through the above process, theprocessing device 110 and/or the mobile device 140 may finally determinethat N (N is an integer greater than 1) sets of first-level candidatereference action data are the second-level candidate reference actiondata. Specifically, N distances (comprehensive comparison results)between the user action data and the N sets of second-level candidatereference action data may be calculated respectively through comparison,and then N probability values may be calculated respectively through theN distances, respectively. The maximum value of the N probability valuesmay be compared with the second preset value, and whether the useraction data includes the second-level candidate reference action datacorresponding to the maximum probability value may be determined. Theprobability that the user action data includes the target actioncorresponding to the i-th second-level candidate reference action datamay be expressed as:

$\frac{1 - {D_{i}/\Sigma_{j}D_{j}}}{\Sigma_{i}\left( {1 - {D_{i}/\Sigma_{j}D_{j}}} \right)}$

where D_(j) denotes the distance (e.g., the aforementioned comprehensiveregularization cost or comprehensive difference degree) between the useraction data and the j-th second-level candidate reference action data.The smaller the distance value between the user action data and thesecond-level candidate reference action data is, the higher theprobability that the user action data includes the target actioncorresponding to the second-level candidate reference action data maybe. For example, by comparing the user action data with threesecond-level candidate reference action data (that is, assuming N=3, thenumbers of the three second-level candidate reference action data are 1,2, and 3, respectively), that a distance between the user action dataand the second-level candidate reference action data 1 is D1 may beobtained, a distance between the user action data and the second-levelcandidate reference action data 2 is D2 may be obtained, and a distancebetween the user action data and the second-level candidate referenceaction data 3 is D3 may be obtained. The smaller the distance valuebetween the user action data and the second-level candidate referenceaction data is, the higher the probability that the user action dataincludes the target action corresponding to the second-level candidatereference action data may be. For example, the probability that the useraction data includes the target action corresponding to the second-levelcandidate reference action data 1 may be determined as follows:

$\frac{1 - {D_{1}/D_{1}} + D_{2} + D_{3}}{\begin{matrix}\begin{matrix}{\left( {1 - {D_{1}/\left( {D_{1} + D_{2} + D_{3}} \right)}} \right) +} \\{\left( {1 - {D_{2}/\left( {D_{1} + D_{2} + D_{3}} \right)}} \right) +}\end{matrix} \\\left( {1 - {D_{3}/\left( {D_{1} + D_{2} + D_{3}} \right)}} \right)\end{matrix}}.$

The probability that the user action data includes the target actioncorresponding to the reference action data 2 may be determined asfollows:

$\frac{1 - {D_{2}/D_{1}} + D_{2} + D_{3}}{\begin{matrix}\begin{matrix}{\left( {1 - {D_{1}/\left( {D_{1} + D_{2} + D_{3}} \right)}} \right) +} \\{\left( {1 - {D_{2}/\left( {D_{1} + D_{2} + D_{3}} \right)}} \right) +}\end{matrix} \\\left( {1 - {D_{3}/\left( {D_{1} + D_{2} + D_{3}} \right)}} \right)\end{matrix}}.$

The probability that the user action data includes the target actioncorresponding to the reference action data 3 may be determined asfollows:

$\frac{1 - {D_{3}/D_{1}} + D_{2} + D_{3}}{\begin{matrix}\begin{matrix}{\left( {1 - {D_{1}/\left( {D_{1} + D_{2} + D_{3}} \right)}} \right) +} \\{\left( {1 - {D_{2}/\left( {D_{1} + D_{2} + D_{3}} \right)}} \right) +}\end{matrix} \\\left( {1 - {D_{3}/\left( {D_{1} + D_{2} + D_{3}} \right)}} \right)\end{matrix}}.$

At this time, the calculated maximum value of the three probabilityvalues may be compared with the second preset value. When the maximumprobability value is greater than the second preset value, it isdetermined that the user action data includes the target actioncorresponding to the second-level candidate reference action data n, andthe second-level candidate reference action data n is the second-levelcandidate reference action data corresponding to the maximum probabilityvalue. When the maximum probability value is less than the second-levelpreset value, it is determined that the user action data that does notinclude the target action corresponding to the reference actiondatabase.

In operation 530, after determining the target action, a content relatedto the target action may be sent to the user.

Specifically, after the user's action is identified, the action of theuser's motion may be monitored, and the monitored information may besent to the user. Monitoring the action of the user's motion may includemonitoring the information related to the user's action. In someembodiments, the information related to the user's action may includeone or more of a user action type, an action quantity, the actionquality (e.g., whether the user action meets a standard), an actiontime, or the like. The action type may refer to the fitness action thatthe user takes when exercising. In some embodiments, the action type mayinclude, but is not limited to, one or more of seated chest clamping,squats, deadlifts, planks, running, swimming, or the like. The actionquantity may refer to the count of actions performed during the user'smotion. For example, the user may perform 10 seated chest clampingduring the user's motion, and the 10 times is the action quantity. Theaction quality may refer to a standard degree of the fitness actionperformed by the user relative to a standard fitness action. Forexample, when the user performs a squat action, the processing device110 may determine the action type of the user's action based on thefeature information of an action signal (the EMG signal and the gesturesignal) corresponding to a specific muscle position (the gluteusmaximus, the quadricep, etc.), and determine the action quality of theuser's squat action based on an action signal of the standard squataction. The action time may refer to a time corresponding to each of oneor more action types of the user or a total time of the motion process.

To sum up, the method and system 100 for determining a target actionprovided by the present disclosure may obtain action data during theuser's motion, and then the action data may be compared with referenceaction data marked with an action content, so that whether the user'smotion includes a target action that is the same as the reference actionmay be identified. The method and system may perform target actionidentification on the user action data without knowing (it is not knownwhether the user has performed the action of the annotation type andwhen the action of the annotation type has been performed) what actionthe user has performed, and send the content related to the targetaction to the user after determining the target action. Through theabove technical solutions, the present methods and systems have higherintelligence than traditional methods and systems, and improve userexperience.

Having thus described the basic concepts, it may be rather apparent tothose skilled in the art after reading this detailed disclosure that theforegoing detailed disclosure is intended to be presented by way ofexample only and is not limiting. Various alterations, improvements, andmodifications may occur and are intended to those skilled in the art,though not expressly stated herein. These alterations, improvements, andmodifications are intended to be suggested by this disclosure and arewithin the spirit and scope of the exemplary embodiments of thisdisclosure.

Moreover, certain terminology has been configured to describeembodiments of the present disclosure. For example, the terms “oneembodiment,” “an embodiment,” and/or “some embodiments” mean that aparticular feature, structure, or characteristic described in connectionwith the embodiment is included in at least one embodiment of thepresent disclosure. Therefore, it is emphasized and should beappreciated that two or more references to “an embodiment,” “oneembodiment,” or “an alternative embodiment” in various portions of thisspecification are not necessarily all referring to the same embodiment.Furthermore, the particular features, structures or characteristics maybe combined as suitable in one or more embodiments of the presentdisclosure.

1. A method for identifying a user action, comprising: obtaining useraction data collected from a plurality of measurement positions on auser, the user action data corresponding to an unknown user action;identifying, based on at least one set of target reference action data,that the user action includes a target action when obtaining the useraction data, the at least one set of target reference action datacorresponding to the target action; and sending information related tothe target action to the user.
 2. The method of claim 1, wherein theidentifying that the user action includes a target action comprises:obtaining a plurality of sets of candidate reference action data,wherein each set of candidate reference action data corresponds to atleast one reference action; performing a two-level screening operationon the plurality of sets of candidate reference action data based on theuser action data, the two-level screening operation including acombination of a difference degree-based screening operation and aprobability-based screening operation; and determining that the useraction includes the target action based on a result of the two-levelscreening operation.
 3. The method of claim 1, wherein the identifyingthat the user action includes a target action comprises: obtaining aplurality of sets of reference action data, wherein each set ofreference action data corresponds to at least one reference action;selecting each set of reference action data in turn from the pluralityof sets of reference action data as candidate reference action data;determining at least one difference degree by comparing at least onesegment of action identification sub-data of the candidate referenceaction data with the corresponding user action sub-data segment bysegment; and determining a comprehensive difference degree by weightingand summing the at least one difference degree.
 4. The method of claim3, wherein, each set of reference action data includes M pieces ofreference action sub-data, each piece of the reference action sub-dataincludes at least one segment of action identification sub-data, and Mis an integer greater than 1; action identification sub-data of the Mpieces of reference action sub-data form integral action identificationdata, and each segment of action identification sub-data corresponds toat least a portion of the reference action on at least one measurementposition of the plurality of measurement positions.
 5. The method ofclaim 3, wherein the determining at least one difference degree bycomparing at least one segment of action identification sub-data of thecandidate reference action data with the corresponding user actionsub-data segment by segment comprises: selecting a sliding window with apreset length on each piece of the action identification sub-data, thesliding window including a data segment of the user action datacollected in a preset time interval; and for the sliding window at acurrent moment, determining the difference degree between the datasegment and the corresponding action identification sub-data.
 6. Themethod of claim 5, wherein the identifying that the user action includesthe target action further comprises: determining that a value of thecomprehensive difference degree is greater than a first preset value;and sliding the sliding window to a next data segment with a preset stepsize, and repeating the comparison.
 7. The method of claim 6, wherein adata collection time length corresponding to the data segment in thesliding window is negatively correlated with a user action speed.
 8. Themethod of claim 7, wherein the preset step size satisfies one or morefollowing conditions: the preset step size is positively correlated witha magnitude of a value of the comprehensive difference degree at aprevious moment; and the preset step size is positively correlated witha variation trend of the value of the comprehensive difference degree.9. The method of claim 5, wherein the data segment comprises a pluralityof user action data points; and the determining at least one differencedegree by comparing at least one segment of action identificationsub-data of the candidate reference action data with the correspondinguser action sub-data segment by segment comprises: selecting a targetcomparison data interval from the action identification sub-data,wherein the target comparison data interval includes a plurality ofidentification data points, adjusting the data segment according to aplurality of scales to obtain a plurality of adjusted data segments,determining a difference degree between the action identificationsub-data and each adjusted data segment of the plurality of adjusteddata segments respectively, and determining a minimum difference degreeamong the difference degrees between the action identification sub-dataand the plurality of adjusted data segments.
 10. The method of claim 5,wherein the determining at least one difference degree by comparing atleast one segment of action identification sub-data of the candidatereference action data with the corresponding user action sub-datasegment by segment comprises: determining a distance matrix [D_(ij)],wherein D_(ij) denotes a distance between an i-th data point of a targetcomparison data interval and a j-th data point of the data segment;determining a shortest distance path of the distance matrix, wherein theshortest distance path satisfies: a start point of the shortest distancepath being in the first line of the [D_(ij)], two adjacent points on theshortest distance path being adjacent in the distance matrix, a nextpoint on the shortest distance path being to the right, below or rightbelow a previous point, an end point of the shortest distance path beingin a last line of the [D_(ij)], and the shortest distance path having asmallest regularization cost, wherein the regularization cost isdetermined by distances of points on the corresponding shortest distancepath of the distance matrix; and the difference degree being related tothe regularization cost.
 11. The method of claim 10, wherein if thefirst data point of the data segment is determined to be a data pointwhere the user action starts, the start point of the shortest distancepath is a distance D₁₁ between the first point of the data segment andthe first point of the target comparison data interval.
 12. The methodof claim 10, wherein if the last data point of the data segment isdetermined to be the data point where the user action ends, the endpoint of the shortest distance path is a distance D_(mn) between thelast point of the data segment and the last point of the targetcomparison data interval.
 13. The method of claim 3, wherein theidentifying that the user action includes the target action furthercomprises: selecting N pieces of second-level candidate reference actiondata from the plurality of sets of reference action data, a value of thecomprehensive difference degree of the second-level candidate referenceaction data being less than a first preset value, and N being an integergreater than 1; calculating N distances between the user action data andthe N pieces of second-level candidate reference action datarespectively; calculating N probability values based on the N distancesrespectively; selecting the second-level candidate reference action datawhose probability value is greater than a second preset value as thetarget reference action data; and determining a reference actioncorresponding to the target reference action data as the target action.14. A system for identifying a user action, comprising: at least onestorage medium, the at least one storage medium storing at least oneinstruction set for obtaining user action data during the user' motion;and at least one processor in communication with the at least onestorage medium, wherein when the system is running, the at least oneprocessor reads the at least one instruction set and executes the methodincluding: obtaining user action data collected from a plurality ofmeasurement positions on a user, the user action data corresponding toan unknown user action; identifying, based on at least one set of targetreference action data, that the user action includes a target actionwhen obtaining the user action data, the at least one set of targetreference action data corresponding to the target action; and sendinginformation related to the target action to the user.
 15. The system ofclaim 14, wherein the identifying that the user action includes a targetaction comprises: obtaining a plurality of sets of candidate referenceaction data, wherein each set of candidate reference action datacorresponds to at least one reference action; performing a two-levelscreening operation on the plurality of sets of candidate referenceaction data based on the user action data, the two-level screeningoperation including a combination of a difference degree-based screeningoperation and a probability-based screening operation; and determiningthat the user action includes the target action based on a result of thetwo-level screening operation.
 16. The system of claim 14, wherein theidentifying that the user action includes a target action comprises:obtaining a plurality of sets of reference action data, wherein each setof reference action data corresponds to at least one reference action;selecting each set of reference action data in turn from the pluralityof sets of reference action data as candidate reference action data;determining at least one difference degree by comparing at least onesegment of action identification sub-data of the candidate referenceaction data with the corresponding user action sub-data segment bysegment; and determining a comprehensive difference degree by weightingand summing the at least one difference degree.
 17. The system of claim16, wherein, each set of reference action data includes M pieces ofreference action sub-data, each piece of the reference action sub-dataincludes at least one segment of action identification sub-data, and Mis an integer greater than 1; action identification sub-data of the Mpieces of reference action sub-data form integral action identificationdata, and each segment of action identification sub-data corresponds toat least a portion of the reference action on at least one measurementposition of the plurality of measurement positions.
 18. The system ofclaim 16, wherein the determining at least one difference degree bycomparing at least one segment of action identification sub-data of thecandidate reference action data with the corresponding user actionsub-data segment by segment comprises: selecting a sliding window with apreset length on each piece of the action identification sub-data, thesliding window including a data segment of the user action datacollected in a preset time interval; and for the sliding window at acurrent moment, determining the difference degree between the datasegment and the corresponding action identification sub-data.
 19. Thesystem of claim 16, wherein the identifying that the user actionincludes the target action further comprises: selecting N pieces ofsecond-level candidate reference action data from the plurality of setsof reference action data, a value of the comprehensive difference degreeof the second-level candidate reference action data being less than afirst preset value, and N being an integer greater than 1; calculating Ndistances between the user action data and the N pieces of second-levelcandidate reference action data respectively; calculating N probabilityvalues based on the N distances respectively; selecting the second-levelcandidate reference action data whose probability value is greater thana second preset value as the target reference action data; anddetermining a reference action corresponding to the target referenceaction data as the target action.
 20. A non-transitory computer readablemedium, comprising at least one set of instructions for identifying auser action, wherein when executed by at least one processor of acomputing device, the at least one set of instructions direct the atleast one processor to perform operations including: obtaining useraction data collected from a plurality of measurement positions on auser, the user action data corresponding to an unknown user action;identifying, based on at least one set of target reference action data,that the user action includes a target action when obtaining the useraction data, the at least one set of target reference action datacorresponding to the target action; and sending information related tothe target action to the user.